Shape from Sheen

Introduction Glossy surfaces are everywhere. If you stand in a typical bathroom and look around, you'll see that most of the surfaces surrounding you are lustrous or glossy. Glazed tiles, gleaming faucets, polished basins and plastic shampoo bottles are all peppered with specular highlights. Given that highlights are so common it seems plausible that the visual system might somehow be able to make use of them in the estimation of three-dimensional shape. Indeed, it is now well established that specular reflections aid shape perception in and Kappers, 1997), and in fact are sufficient on their own to confer a vivid sense of 2004). This is demonstrated in Figure 1, which shows a computer-generated image of a perfectly specular (i.e. mirrored) surface. Despite the fact that the image contains no shading, texture or other cues to 3D shape, we nevertheless have a vivid impression of its 3D shape. Shape from sheen 2 Figure 1. A computer-generated image of a perfectly specular surface reflecting a natural environment. Even though the image contains no stereo, motion, shading or texture cues, we still have a vivid sense of the object's 3D shape. This chapter is about the image information that can be used to recover 3D shape from specular reflections. This chapter is about how the visual system could recover 3D shape from specularities, from a theoretical point of view. We start by discussing some of the problems that specular reflections seem to pose to the visual system. We then argue that the visual system can use the way that reflections are distorted across the surface to recover some important constraints on 3D shape. We show that the pattern of distorted reflections remains surprisingly even when different scenes are reflected in the surface of the object. The key idea presented in this chapter is that relatively simple image measurements can sometimes carry unexpectedly powerful information about the 3D world. Shape from sheen 3 The optical misbehaviour of mirrors Glossy surfaces exhibit a number of behaviours that make them potentially highly problematic for human vision. Specular reflections are quite paradoxical. On the one hand they are one of the simplest interactions that light can have with matter. On the other hand they create distorted, impenetrable images of the world on the surface of an object, which writhe and flow over the surface as the object is moved. We will now discuss some of the qualities …